Scaling Up Simultaneous Diagonalization

Simultaneous matrix diagonalization is a key subroutine in many machine learning problems, including blind source separation and parameter estimation in latent variable models. Here, we extend joint diagonalization algorithms to low-rank and asymmetric matrices and also provide extensions to the perturbation analysis of these methods. Our results allow joint diagonalization to scale to larger problem sizes and to new domains; we give a survey of such applications and report improvements relative to the state-of-the-art on a latent variable learning task. We hope that our results will demonstrate the usefulness and versatility of joint diagonalization as a tool in optimization and machine learning.